Student Rotation Changes
Preliminary Analysis
Kate Weber
Feburary 2021
Background and Significance
- Rotation Change Issues
- Learning Health System Approach
- Graph Analysis Approach
- Why is this new?
Objective
Um. Unclear right now. :)
Materials and Methods
For this analysis, we transcribed the data from the school years 2017-18, 2018-19, and 2019-20 from scanned copies of the original paper request into a spreadsheet. We used a python script to load the data into a Neo4j graph database. We classified student rotation changes by school year; by the class of the student making the change; the type of rotation being changed; the reason for change; the student’s membership in the ITDP program
[Paragraph for collecting data about effort associated with changes before and after migration to electronic forms]
Results
Network Visualization
I think in the normal course of things, this would be a theme that develops after the kind of ordinary descriptive work is done (ideas below). But since that is the part that really illuminated this conversation, here it is:
Each colour represents one graduating class - 2018 in blue, and 2019, 2020, and 2021 in green, purple, and orange proceeding clockwise. 2018 and 2021 each are partial datasets - we have only 2018’s D4 year and 2021’s D3 year in the dataset at present. For the network analysis, we omitted changes that did not require another student substitution, as the primary motivator was understanding the cascading impacts of student-student rotation changes. [kgw - this can be redone to include/contrast CBCE changes if necessary]
Each student is shown as a circle (node) and each change request is a grey line (edge) connecting students. The edges in this network are directed, that is, they point from the student who requested the change to the student who swapped in for them. This does not always reflect who actually initiated the change, as a few request forms indicate that the requestor is filing the form on behalf of their fellow student, but that is rare enough that the network is worth analyzing in directed form.
We can visualize networks in light of the properties of the nodes or of the edges. An alternative view of the same plot can show different student attributes of interest. For example, this view shows the same class structure but instead indicates which students are ITDP students:
D4 Change Networks
Our first analysis of the students’ changes was over the 2019-2020 school year. In this view and the following D4 views, nodes are coloured darker when a student has been involved in more changes.
We noted a relatively open structure with few chains of changes and initially surmised that although changes happen, they are not generally complex cascades of change. The 2018-19 and 2017-18 school years, however, are much more complex networks of change among the students:
To investigate this further, we ran six graph algorithms over these networks to assess their structure:
- Network Density
- Centrality
- Degree Centrality
- Betweenness Centrality
- PageRank Centrality
- Community Detection
- Louvain Community Detection
- Label Propagation
Networks are measured in terms of the number of nodes (n) and edges (m). A key measure of any node’s connectedness is its degree, the number of edges between it and other nodes. The mean of a network’s node degrees is noted as c.
Network Density
A simple measure of network complexity is density:
\[ \rho =\frac{m}{\binom{n}{2}} = \frac{2m}{n(n-1)} = \frac{c}{n-1} \] where \(0 \le \rho \le 1\). The D4 network densities are impacted by the relatively large number of students who do not trade with other students at all. The visual overall impression that 2019-2020 is different from the previous years is borne out by examining the networks’ density:
Centrality
In addition to examining networks as a whole, we can look at individual nodes and observe how key they are to the network with a variety of measures. Degree simply counts the number of edges associated with each node. Figures [foo] and [bar] above are coloured by degree and show an increase in the number of students with a larger number of changes.
This figure is a histogram showing how many students have any given degree. It shows that D3s on the whole have a large number of students who make no changes at all (degree 0) and that in 2019-20, a larger than usual number of D4 students had a smaller degree than in previous years. Additionally, in 2019-20, there were no D4s who had more than six changes, whereas a few “frequent fliers” in prior years were involved in frequent (or complex) changes.
PageRank
Designed for use by Google for determining whether a web site should be prioritized in search results, PageRank has become a widely-used indicator of centrality in computing, transportation, mathematical, and social networks. PageRank measures the importance of a node in a network in terms of how connected it is but also considers the importance of the nodes pointing to it. For the purposes of this analysis, PageRank shows students who have taken other students’ rotations, while prioritizing those partners who themselves have taken rotations.
KGW: This is the helpers’ circle
Betweenness
Betweenness centrality points out nodes that are on the shortest path between other nodes in the network. In theory, betweenness shows nodes that are involved in more communication - whose absence is most likely to disrupt the network. Social studies use betweeness to examine who may exert influence by providing “social glue” or mediating messaging between other members of the community.
KGW: these are the go-betweens
Comparing PageRank and Betweenness Visualizations
17-18 PageRank 
17-18 Betweenness 
18-19 PageRank 
18-19 Betweenness 
19-20 PageRank 
19-20 Betweenness 
Summary of Centrality Measures
We do not see statistically significant variance for any of these centrality measures for any of the centrality measures among students from class to class or from year to year. Betweenness is notably unhelpful for considering the D3 students (not visualized above) because their rotation change networks are significantly less complex than they are for D4s. The very loosely-connected network for the Class of 2021 (Orange in figure [x the big overview above]), for whom we only have D3 data at present is an example.
Tabulated Network Metrics
| school_year | class | nodeCount | relationshipCount | density | minDegree | maxDegree | meanDegree |
|---|---|---|---|---|---|---|---|
| 17-18 | D3 | 125 | 94 | 0.006 | 0 | 7 | 0.752 |
| 17-18 | D4 | 133 | 283 | 0.016 | 0 | 11 | 2.128 |
| 18-19 | D3 | 131 | 125 | 0.007 | 0 | 5 | 0.954 |
| 18-19 | D4 | 125 | 295 | 0.019 | 0 | 12 | 2.360 |
| 19-20 | D3 | 128 | 132 | 0.008 | 0 | 5 | 1.031 |
| 19-20 | D4 | 131 | 158 | 0.009 | 0 | 6 | 1.206 |
Community Detection
Network algorithms can detect community structure - likely social or technical communities dependent on connections with each other. We passed two different community detection algorithms over the D4 rotation swaps - this is the output of an algorithm called Louvain community detection.
[kgw - I’ve anonymized all this data really heavily. I’m extremely curious to know whether the community structures detected actually reflect anything about the structures of the class’ cliques]
17-18 Louvain Communities 
18-19 Louvain Communities 
19-20 Louvain Communities 
Cross-Tabulating Reason, Class, School Year, and Rotation
[Describe kinds of changes and overall patterns between classes and from one school year to the next]
This table indicates how many changes were requested by students in each class, subdivided by whether the request was strictly for time off from CBCE or whther it required a swap with another student.
17-18 |
18-19 |
19-20 |
||||
|---|---|---|---|---|---|---|
| 3 (N=104) |
4 (N=331) |
3 (N=138) |
4 (N=351) |
3 (N=140) |
4 (N=208) |
|
| swap_type | ||||||
| CBCE Time Off | 10 (9.6%) | 48 (14.5%) | 13 (9.4%) | 56 (16.0%) | 8 (5.7%) | 50 (24.0%) |
| Student Swap | 94 (90.4%) | 283 (85.5%) | 125 (90.6%) | 295 (84.0%) | 132 (94.3%) | 158 (76.0%) |
Reasons
Student switch rotations for a variety of reasons. The school’s policy allows for [fill in what the school’s policy allows for]. 
Alternative view of Change reasons - absolute numbers rather than proportionally
Or tabular
17-18 |
18-19 |
19-20 |
||||
|---|---|---|---|---|---|---|
| D3 (N=104) |
D4 (N=331) |
D3 (N=138) |
D4 (N=351) |
D3 (N=140) |
D4 (N=208) |
|
| reason | ||||||
| Patient Care | 13 (12.5%) | 41 (12.4%) | 24 (17.4%) | 35 (10.0%) | 32 (22.9%) | 26 (12.5%) |
| Family | 6 (5.8%) | 43 (13.0%) | 5 (3.6%) | 72 (20.5%) | 5 (3.6%) | 32 (15.4%) |
| Rotation Conflict | 16 (15.4%) | 57 (17.2%) | 2 (1.4%) | 65 (18.5%) | 14 (10.0%) | 8 (3.8%) |
| Other | 12 (11.5%) | 35 (10.6%) | 2 (1.4%) | 35 (10.0%) | 9 (6.4%) | 23 (11.1%) |
| Interview | 0 (0%) | 40 (12.1%) | 4 (2.9%) | 25 (7.1%) | 0 (0%) | 42 (20.2%) |
| SoD Travel | 30 (28.8%) | 20 (6.0%) | 32 (23.2%) | 4 (1.1%) | 24 (17.1%) | 1 (0.5%) |
| SoD Conflict | 1 (1.0%) | 16 (4.8%) | 16 (11.6%) | 21 (6.0%) | 18 (12.9%) | 27 (13.0%) |
| Unknown | 0 (0%) | 22 (6.6%) | 20 (14.5%) | 21 (6.0%) | 12 (8.6%) | 24 (11.5%) |
| Wedding | 13 (12.5%) | 24 (7.3%) | 6 (4.3%) | 5 (1.4%) | 11 (7.9%) | 8 (3.8%) |
| Conference | 9 (8.7%) | 12 (3.6%) | 11 (8.0%) | 20 (5.7%) | 9 (6.4%) | 4 (1.9%) |
| Health | 0 (0%) | 14 (4.2%) | 5 (3.6%) | 34 (9.7%) | 0 (0%) | 8 (3.8%) |
| Travel | 4 (3.8%) | 7 (2.1%) | 11 (8.0%) | 14 (4.0%) | 6 (4.3%) | 5 (2.4%) |
Rotations Impacted
I could really use some informed input about whether any of this information is actually useful
Or tabular
17-18 |
18-19 |
19-20 |
||||
|---|---|---|---|---|---|---|
| D3 (N=104) |
D4 (N=331) |
D3 (N=138) |
D4 (N=351) |
D3 (N=140) |
D4 (N=208) |
|
| rotation | ||||||
| CBCE | 4 (3.8%) | 105 (31.7%) | 8 (5.8%) | 152 (43.3%) | 0 (0%) | 77 (37.0%) |
| PAES | 1 (1.0%) | 109 (32.9%) | 0 (0%) | 89 (25.4%) | 0 (0%) | 67 (32.2%) |
| Other | 12 (11.5%) | 40 (12.1%) | 10 (7.2%) | 33 (9.4%) | 13 (9.3%) | 30 (14.4%) |
| Unknown | 0 (0%) | 23 (6.9%) | 20 (14.5%) | 15 (4.3%) | 14 (10.0%) | 22 (10.6%) |
| Oral Surgery | 0 (0%) | 0 (0%) | 22 (15.9%) | 28 (8.0%) | 33 (23.6%) | 10 (4.8%) |
| Radiology | 24 (23.1%) | 0 (0%) | 22 (15.9%) | 0 (0%) | 31 (22.1%) | 0 (0%) |
| RRR | 22 (21.2%) | 0 (0%) | 26 (18.8%) | 0 (0%) | 17 (12.1%) | 0 (0%) |
| Ortho | 18 (17.3%) | 0 (0%) | 16 (11.6%) | 2 (0.6%) | 6 (4.3%) | 2 (1.0%) |
| D2 Mentor | 0 (0%) | 14 (4.2%) | 0 (0%) | 22 (6.3%) | 0 (0%) | 0 (0%) |
| OS | 10 (9.6%) | 24 (7.3%) | 0 (0%) | 0 (0%) | 0 (0%) | 0 (0%) |
| Peds | 13 (12.5%) | 0 (0%) | 14 (10.1%) | 0 (0%) | 0 (0%) | 0 (0%) |
| Implant | 0 (0%) | 16 (4.8%) | 0 (0%) | 10 (2.8%) | 0 (0%) | 0 (0%) |
| Pedo | 0 (0%) | 0 (0%) | 0 (0%) | 0 (0%) | 26 (18.6%) | 0 (0%) |
Temporal aspect
Early analysis leaned on an assumption that the 2019-2020 school year’s relative simplicity was driven by the system shutdowns that accompanied the COVID pandemic in the spring of 2020. On further consideration, though, the vast majority of the school year had passed before the pandemic really impacted operations - Michigan suspended school and limited clinical operations on March 13, 2020.
No rotation changes were approved in April 2020. That year also had almost no D4 SoD Travel changes.
Costs Associated with Changes
This needs to be built out with a longer look into the actual problem - it's going to drive how we design any interventions. The whole problem right now is strictly anecdotal and needs to be better-quantified beyond "how many changes? Components include identifying all parties who get tangled up in a change; determining whether changes have different levels of impact (what's the work associated wtih a CBCE change vs a regular clinical rotation?) and then cost it out in time expended, time duration, and what parties are affected.
Discussion
Why was 2020 different?
Is it actually different?
Is the system just really brittle? A fair bit of the 2018-19 chaos can be traced back to one pregnancy. Is that enough to make any class really squirrelly
Are there things we can learn?
Will the move to electronic forms change change behaviour?
Did it save effort and/or money? If it’s easier for students to deal with their rotation paperwork, will they file more of it?
Who changes and why?
Are changes preventable?
How can we drill into reasons for change and preventing them without unfairly profiling students?
Are there conversations we can have about community structure and DEI? About ensuring students have access to the support systems they need? It didn’t really look to me as though the ITDP students were all off in their own corner…
COVID impact
limited - but did cancel late-semester SOD travel
Limitations
Inconsistent reason coding Limited number of years to work with Difficulty sorting out COVID impact if any